Skip to content

perf(hmr): consolidate fast incremental updates and lazy-runtime correctness#14772

Open
matthewdavis-oai wants to merge 29 commits into
web-infra-dev:mainfrom
matthewdavis-oai:matthewdavis-oai/hmr-outgoing-cache
Open

perf(hmr): consolidate fast incremental updates and lazy-runtime correctness#14772
matthewdavis-oai wants to merge 29 commits into
web-infra-dev:mainfrom
matthewdavis-oai:matthewdavis-oai/hmr-outgoing-cache

Conversation

@matthewdavis-oai

@matthewdavis-oai matthewdavis-oai commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Fixes #14768. Fixes #14769. Fixes #14767. Fixes #14766. Fixes #14790.
Fixes #14791. Fixes #14792. Fixes #14793. Refs #14401, #14411, #14682, #14757, #14765, #14770,
#14779, #14789, #12904, and #6633.

This consolidates the trace-backed HMR, chunk-graph, Copy/watch, lifecycle,
and lazy-compilation correctness work into one reviewable stack. The
performance paths add no configuration switches and fall back whenever the
provenance required for a safe fast path is unavailable. The lazy-HMR port
retains its narrowly scoped, documented preserveDisposedModuleFactories
apply option and exposes typed Compilation.watchInvalidationKind so
integrations do not need process-global Symbols. The lazy/runtime source is
ported from #14753 with the original co-author attribution retained.

What each part fixes

Area Cause Fix and correctness boundary
Chunk-graph reuse Async outgoings were compared with the root block; source order/filtering differed from CodeSplitter::prepare; existing zero-outgoing leaves looked like new modules. Share connection preparation/filter/order logic, compare root and async blocks independently, retain group-option/nested-block bailouts, and reuse only an existing dependency-less root already present in the recovered chunk graph.
Dynamic/global entry correctness Reuse checked only entry names, so changed dynamic entry options, ordered roots, includes, or request attribution could leave an installed chunk graph stale while another entry kept the modules alive. Function-valued filename/public-path options were rewrapped every generation, causing false misses or stale closures under blind reuse. Compare all topology-affecting entry fields, ordered/deduplicated global and named roots, original requests, and compilation-scoped includes. Refresh current filename/public-path functions on a safe hit and conservatively disable incremental chunk assets for full-hash/function filenames. Duplicate shared roots still reuse.
JS entry/include bridge correctness Dependency reuse ignored the raw resolution context; equal relative requests from different contexts could silently omit a module. Unnamed entries and includes were marked as ordinary entries and could be replaced by lazy proxies that never execute. Key dependency identity by raw context/request/name/layer, preserve dependency IDs across output-option changes, mark unnamed entries and all includes eager, and retain ordinary named-entry laziness.
Stable async identity Moving an unchanged import() changed its source-location-based block ID and forced a full graph rebuild. Keep semantic dependency/modifier identity, remove source-location identity, and assign collision-safe occurrence suffixes for repeated blocks.
HMR process_assets Every recorded chunk was linearly searched, all memberships were cloned/scanned, and an ordinary edit revisited unchanged chunks. Index chunk IDs once, calculate actually removed modules once, and use ChunkSetHashes to skip unchanged surviving chunks. Runtime-membership changes and disabled incremental chunk assets always take the full path; an edited module that moves chunks is still emitted to its installed old chunk.
Copy/watch correctness and reuse Static patterns rescanned unchanged assets and cache hits allocated/logged once per pattern; an empty but valid lazy-watch rebuild was indistinguishable from a provenance-free rebuild and recopied every asset; glob contexts could omit an initially empty sibling; ancestor events could leave cached files stale; equal-priority forced copies used an unstable priority-only result sort, so later configured patterns or later results within one glob could lose nondeterministically; Windows could emit backslash-separated asset keys. Linux stale-watch cleanup was quadratic and overlapping registrations could silently remove child watches. Prepartition cache hits/misses under a dense indexed cache, carry the existing lazy-watch provenance to the native compilation so an empty lazy generation can safely reuse its static result, report one structured hit/total counter, run only misses, avoid redundant source/permission allocations, register the stable glob base and invalidate symmetric path overlap, sort the smaller pattern list by (priority, pattern index) while preserving each result vector's enumeration order, normalize emitted asset names to POSIX, and index stale paths/recursive ancestors. Dynamic/callback/template/permission and empty normal/programmatic paths remain conservative.
Lazy compilation/watch provenance A lazy activation could invalidate unrelated compilers, paused watcher batches could be replayed or lost, mixed lazy/file generations could be published stale or misclassified, and prefix matching routed compiler 10 to compiler 1. Target the activated child with exact path/query matching, atomically drain/acknowledge native batches and clear paused Watchpack aggregation, keep the native aggregate callback queue lossless while the executor limits it to one in-flight batch, snapshot typed invalidation provenance per compilation, classify artifact-dependent children as normal, and let normal/file-backed work dominate a single coalesced generation.
Lazy HMR/runtime filename correctness A chunk can become initial during an update, extracted async CSS using [fullhash] could call an undefined hash runtime, and filename-requirement checks repeatedly traversed all reachable chunks. Removed factories may still be needed until an apply transaction settles. Keep referenced async-entry filenames aligned with code generation, derive/cache the JS/CSS full-hash requirement once per compilation (including per-chunk and extracted-CSS templates), preserve disposed factories only for the transaction, and clean parent/child links on every settle/error path.
Native watcher lifecycle The N-API close path asynchronously shared/mutated an &mut FsWatcher; overlapping watch/pause/close was explicitly documented as undefined behavior, the native matrix reproduced an access violation (0xC0000005), immediate rewatch/double-close can hang a local binding stress loop, and a retained prior JS Watcher handle could pause, drain, or close the live reused native watcher. Own the watcher behind an async mutex plus atomic closed state, serialize watch/close, reject post-close use, keep short synchronous operations locked, permit only the owning JS watch generation to pause/drain/close or inject callbacks, acknowledge obsolete aggregates without affecting the live batch, and stress overlapping watch/pause/double-close, stale handles/shims, a 400-file no-pause close race, and immediate recreation.
Native watcher raw context events A new/changed/removed child below a context dependency was collapsed to its parent, the one-slot N-API raw callback queue silently dropped sibling events, and conflating raw paths with compilation aggregates duplicated directory callbacks and inflated invalidation sets. Keep Watchpack-compatible registered/context aggregation and a separate concrete raw projection, deliver concrete child paths to long-lived `watchFileSystem.on('change'
Extracted-CSS HMR ownership The native hmrC.miniCss handler and the loader-level, debounced cssReload() both replaced the same stylesheet, causing nearly two browser requests per edit and a transient duplicate link even with one compiler generation. Let the native handler own replacement when present and retain the loader fallback only for CssExtractRspackPlugin({ runtime: false }); CSS-module locals invalidation and dispose data remain unchanged.
HMR test isolation/runtime disposal Concurrent web/worker hot cases shared process-global console.warn, so a neighboring disposed-module case could capture extra warnings. A removed factory re-required during a preserve transaction created a fresh cached module that stayed hot.active after finalization, allowing retained closures to silently reattach stale parent edges. Give each WebRunner an inheriting scoped console and always restore warning hooks in finally; deactivate the transient module before unlink/cache deletion, and assert its post-idle closure warns and cannot reattach the removed parent.
Compilation dependency deltas Context, missing, and build added iterators incorrectly appended compilation-level file dependencies. Append the corresponding dependency sets and add a focused disjoint-iterator regression. The more aggressive persistent-snapshot filtering proposal in #14765 is deliberately not included because provenance-free invalidation can make it unsound.
Long-session lifecycle ModuleGraphConnectionWrapper and CSS/SRI/Rsdoctor compilation maps retained prior generations. Call the existing connection cleanup and remove all compilation-keyed plugin entries in clear_cache(old_id).

Commit/issue map

The final commits are deliberately separated so the entry bridge, runtime,
watcher/Copy, lazy middleware, and in-flight coalescing changes can be lifted
or reviewed independently.

Latest correctness pass

The final adversarial pass adds two narrow boundaries. First, a legitimate
lazy watch rebuild can have empty modified/removed sets; Copy now reuses its
unchanged static pattern only when explicit lazy provenance reaches the native
compilation. Empty normal invalidation and repeated compiler.run() still
recompute, including a source mutation with its original mtime restored. The
original Rust rebuild API and three-argument binding call remain compatible;
the provenance marker is an optional trailing binding argument and appears on
the Compiler:rebuild Perfetto span. Second, retained watcher handles,
callbacks, and shims from an earlier generation cannot pause, drain, close,
or inject into the current native watcher; obsolete aggregates are still
acknowledged so delivery cannot stall. Long-lived raw listeners continue to
receive concrete created/removed children across cycles.

The focused final matrix is 24/24 across Copy, native generation,
lifecycle/raw-event, and lazy artifact-chain cases. It includes a real
2,000-file activation, empty-normal and repeated-run freshness, native and
Watchpack coalescing, raw bursts, stale handles/shims, and 1,000 lifecycle
races.

Measured evidence

A grouped release-binding HMR fixture with 1,057 modules / 194 emitted cold
chunks / 195 chunks tracked on rebuild (three groups of four leaf edits)
shows:

Phase Baseline median Initial optimized Final consolidated
leaf rebuild 403.33 ms 40.34 ms 30.17 ms
code splitting 28.26 ms 0.66 ms 0.55 ms
chunk hashing 51.55 ms 5.22 ms 4.42 ms
chunk assets 70.00 ms 8.69 ms 5.84 ms
HMR process_assets 10.84 ms 1.14 ms 0.51 ms

The baseline changes 195/195 tracked chunks per edit; the optimized path
changes 2/195, emits exactly two hot-JS assets plus one manifest with the
edited marker, and preserves runtime/module removal behavior. In a separate
stability run, the final binding completes 40/40 measured leaf edits with the exact two-chunk
boundary, approximately a 13x median rebuild reduction for the sparse
fixture.

An independent pre-lazy-consolidation release-binding comparison against the
earlier optimized binding (five groups of four edits, with
emitted-value/removal assertions)
finds additional headroom: leaf median 36.37 -> 29.77 ms (-18.1%), HMR
process_assets 1.13 -> 0.52 ms (-54.0%), rebuild chunk assets
6.80 -> 5.63 ms (-17.2%), hashing 5.14 -> 4.17 ms (-18.9%), and
chunk runtime requirements 2.61 -> 2.00 ms (-23.4%). Cold median is
381.80 -> 365.85 ms (-4.2%) with cold chunk assets
43.22 -> 40.45 ms (-6.4%) and runtime requirements
13.14 -> 11.10 ms (-15.5%); removal improves
114.51 -> 103.18 ms (-9.9%). The small CodSpeed cold-stage signals do
not reproduce in this grouped local comparison.

An idle, bracketed 2.1.4 comparison isolates the earlier four correctness
commits from their preceding combined binding. Leaf/cold/removal medians are
29.41/382.79/101.64 ms -> 30.17/367.59/102.14 ms
(+2.6%/-4.0%/+0.5%), with every rebuild/runtime-loss/removal assertion
passing and phase deltas within roughly 0.2 ms. This does not show a
material regression from the final entry/runtime/watch guards.

For Copy, a matched debug-native watcher A/B with 374 files/29.92 MiB and
three repetitions per group gives static-pattern medians of
65.78 -> 1.69 ms (-97.4%), compilation 135 -> 72 ms (-46.7%), and
edit-to-done 150.34 -> 80.13 ms (-46.7%). The mixed case retains exactly
80 dynamic transform calls and two static-pattern hits, with pattern work
67.23 -> 21.98 ms (-67.3%), compilation 131 -> 95 ms (-27.5%), and
edit-to-done 151.90 -> 105.68 ms (-30.4%).

The empty-lazy Copy boundary has a separate bracketed CI-profile A/B using
2,000 x 16 KiB static assets (about 32 MiB). Every run asserts exactly
2,000 emitted assets, watchInvalidationKind: lazy, and empty modified and
removed sets. The preceding binding records 0/1 Copy hits on 10/10
activations with 320.5 ms median edit-to-done (292--365 ms) and
310.5 ms compiler time (282--354 ms). The final binding records
1/1 on 10/10 and improves those medians to 215 ms
(202--248 ms) and 206 ms (192--235 ms): roughly 105 ms / 33%
saved. Empty normal/programmatic invalidation still recomputes, including a
source mutation whose mtime is restored. The compatible optional provenance
argument adds 0 B to the stripped CI-profile binding; total size remains
69,831,832 B, +49,152 B over main and 2,048 B below the gate.

A grouped release-binding Copy-pattern scale comparison (baseline ->
prepartition candidate -> baseline; three warmups and twelve candidate/twenty-
four pooled-baseline timed rebuilds per scenario) covers
16 x 24 x 80,000 B (~30.72 MB), 128 x 2 x 512 B, and
512 x 1 x 256 B. Every rebuild checks every asset, exact cache-hit count,
and dynamic-transform count. At 512 static patterns, run-pattern medians for
all-hit/one-miss/52-miss improve 1.163/1.380/20.76 ms ->
0.626/0.863/19.94 ms; compiler medians improve 48/41/66 -> 40/38/58 ms
and p95 improves 63/61/100 -> 55/50/75 ms. Dynamic controls retain the
exact 24/2/1 transform calls per rebuild; 512-pattern run-pattern medians
improve 1.494/1.670/21.91 -> 1.005/1.083/21.26 ms. The structured cache
counter was added after this binding and replaces per-hit debug-string
allocation with one standard hit/total record; these timings are deliberately
not attributed to that later logger change.

The two concurrent Copy joins intentionally collect directly into
FuturesOrdered.
With the pinned futures 0.3.32 implementation, join_all switches strategy
at 30 futures and generates both MaybeDone and ordered poll/drop paths;
symbol attribution showed several 7--10 KB Copy-specific instances. Direct
ordered collection preserves concurrency, input order, and successful-sibling
error semantics while removing that duplicate small-path monomorphization.
The synchronous pattern-order/emission step is a small dedicated helper, so it
is not duplicated in the instrumented async state machine. The exact
CI-equivalent, path-trimmed binding is 69,831,832 B stripped versus
69,782,680 B for main (+49,152 B), leaving 2,048 B under the
51,200 B binary-size gate; it contains zero local source/registry paths.

A final bracketed guard run (24 pooled baseline/12 candidate rebuilds)
confirms the extraction does not regress Copy scale: at 512 static patterns,
all-hit/one-miss/52-miss pattern medians are
0.831/0.949/23.72 -> 0.625/0.841/23.63 ms and compiler p95 is
105/72/90 -> 61/63/87 ms; the dynamic control is
1.048/1.187/23.71 -> 0.934/1.095/23.75 ms with exactly one transform.
All asset, cache-hit, and overwrite-order assertions pass; remaining
compiler-median differences are within run noise.

The scale harness also discriminates a pre-existing correctness bug. With 64
interleaved forced copies and priority = index % 5, the configured final
priority-4 pattern is 59 but the priority-only unstable sort emits 44; at 128
and 512 patterns the observed winners are 94 instead of 124 and 309 instead
of 509, across initial and every mixed-cache rebuild. Result-level sorting
still leaves all files within one glob tied: with 64 glob results and 32
interleaved patterns the last enumerated file is 39, but the result sort emits
27 (and larger mixes can emit 0). Sorting the much smaller pattern list by
(priority, pattern_index) and preserving each result vector's order restores
both configured and within-glob precedence. The regression covers initial,
early miss, winner miss, all-hit, and intra-pattern rebuilds.

The stale-watch cleanup path previously compared every current registration
against every stale path. A standalone optimized microbenchmark that retains
half of the registrations and removes the other half shows the indexed,
component-aware path scaling with tree depth instead of the Cartesian product:

stale/current watches previous median indexed median
2,048/4,096 569.08 ms 0.92 ms
4,096/8,192 2,428.60 ms 1.93 ms
8,192/16,384 9,669.48 ms 3.94 ms

This retains recursive-child and near-prefix-sibling semantics; it is not an
inotify-registration benchmark. Native recursive registration still traverses
ignored descendants before filtering events (5,000 ignored top-level
directories produced 10,023 native watches and about 1.25 s ready time,
versus 23 watches/about 0.32 s with Watchpack). That separate scalability
limitation and an ignored-aware registration direction are documented in
#14789.

The native raw-event path has a separate correctness and scale boundary. A
deterministic synchronous burst of 2,048 registered edits delivers
0/2,048 unique concrete paths at the JS raw callback in the recorded
previous one-slot/non-blocking N-API run even though compilation aggregation is
complete; this is callback-queue backpressure, not evidence that the OS watcher
generated zero events. The final watcher delivers every
expected path/kind for 2,048 context-only creates, removes, and delayed
create-after-delete events while keeping the aggregate at exactly the watched
context. In the captured final run at 2,048/8,192/16,384 registered
edits, all paths arrive with 2,747/13,985/22,976 raw callbacks,
35.0/179.0/447.6 ms delivery, and approximately 3.0/9.5/19.3 MiB
whole-process RSS deltas while a Rust check was active. An earlier idle run
records 33.6/85.1/146.4 ms delivery. Extra callbacks above the edit count
include explicit triggers and OS notifications; no expected event is lost. The
normal trigger
stores its single raw event inline and scanner batches retain one allocation.

A dense runtime fixture with 192 entries, 192 shared async chunks, and about
37,000 import edges separates filename correctness from the remaining cold
compile cost. In clean, bracketed output-full-hash runs, the preceding/final
binding ranges are 3,746--3,879 / 3,826--3,861 ms wall time; module graph
is 2,715--2,810 / 2,741--2,825 ms, code splitting
351--378 / 357--368 ms, runtime requirements 210--227 / 202--226 ms,
hashing 248--249 / 246--260 ms, and chunk assets
72 / 71--75 ms. Async-full-hash and cache-group overrides show the same
throughput-neutral pattern, with every filename/runtime-helper assertion
passing. Across two dense runs, the four CSS factory and HMR/URL parser hooks
fire 74,112 times in total; their combined close-span time is about
293 ms. The useful next upstream target is repeated factory/parser-hook
dispatch and resource scheduling in dense graphs, not another
runtime-template traversal.

An earlier optimized-stack run in a large route-graph browser harness
completed 20/20 true HMR/SSR updates with one web/node generation per
edit, no steady CSS or lazy requests, and no navigation. Visible p50 was
4.20--4.29 s for the first 15 measured edits and 5.17 s for the final
five as the process grew; corresponding browser compilation medians were
2.89/3.26/3.98 s. This predates the final ChunkSetHashes, terminal-leaf,
and Copy/watcher correctness commits, so it is phase attribution and a useful
target, not a claim about the exact final SHA. Remaining trace targets are
module-graph scheduling, asset materialization/source maps, repeated
SplitChunks work (#14770), and per-generation memory growth.

The large multi-entry leaf trace makes that boundary concrete: the edited TSX
transform itself is only 4--6 ms. In the historical pre-alias trace, the
development-server root-stylesheet loader costs ~0.92--1.11 s; removing
that redundant server graph eliminates the cost. Remaining warm compiler
costs are SplitChunks (~0.75--1.40 s), runtime requirements
(~0.63--0.95 s), and process assets (~0.2 s).
In the dense cold fixture, module graph is about 2.76 s/~70%, code
splitting ~0.36 s, runtime requirements ~0.22 s, and hashing
~0.26 s; four parser/factory hooks fire 74,112 times across two runs
(about 146 ms/run combined). The useful next targets are hook
dispatch/resource scheduling and asset materialization, not another
filename-template traversal.

Related open upstream work is intentionally not included in this stack:

  • feat(watcher): populate native watcher file/context time info entries #14411 is a draft for native-watcher time-info parity. It populates
    fileTimeInfoEntries/contextTimeInfoEntries on the native callback and
    leaves Watcher.getInfo() and the undelayed change time out of scope; it is
    separate from the lifecycle/coalescing fixes here.
  • feat(hmr): emit css.c / css.r in the hot-update manifest for precise CSS updates #14682 tracks precise native/extracted CSS HMR (css.c/css.r and
    miniCss namespaces), eliminating stylesheet re-fetch/reparse on a
    JavaScript-only update and handling CSS add/remove. That browser-runtime
    work is separate from this stack's compiler-phase costs; the large-route
    run above already observes zero steady CSS requests. A clean trial merge
    with this stack has no source conflicts, but its CI-profile binding is
    69,852,312 B stripped (+69,632 B over main), exceeding the
    51,200 B binary-size gate by 18,432 B; keeping it independently
    reviewable avoids duplicating its 29 attributed commits. The loader guard
    here remains compatible and prevents a second owner whenever a proxy module
    is disposed.
  • perf: reduce source map copying #14401 is a draft, production-oriented source-map generation optimization
    that reduces mappings allocation/copying. It does not address the
    incremental source-map cache/re-emission pass identified in this trace,
    but is useful adjacent profiling context.

Committed regressions and validation

The Linux chromium-incremental E2E run on df63754 exposed the extracted-
CSS double-owner race: the runtime hmrC.miniCss handler and loader-level
cssReload() could both swap the same stylesheet, leaving two links after an
edit. A four-worker, ten-repeat browser matrix now exercises 20 edits x 10
repetitions for each runtime mode (400 edits total), asserts one link
and exactly one unique/cache-busted HMR stylesheet response per edit, and
passes both modes. The unguarded baseline performs one compiler rebuild per
edit but produces 392 unique stylesheet responses and 38 post-
assertion two-link observations for 200 edits; the guarded run produces
exactly 200/200 with zero duplicate links. All 400 guarded updates compile
in 11--53 ms (p50 20 ms, p95 32 ms). This fixes the duplicate-
request/link boundary without claiming the precise JavaScript-only CSS work
elimination tracked in #14682.

  • async-outgoing incremental-watch fixture: eight steps cover leaf reuse,
    async chunk rename/retarget/add/remove misses, a same-module
    source-position move, a dependency-less terminal-leaf edit, synchronous
    order/side effects, and emitted async values; the existing pre-order-index
    leaf fixture is re-enabled;
  • four Rust async-identity cases cover location moves, duplicate blocks,
    unrelated insertion, and occurrence-suffix collisions;
  • four HMR-process-assets cases cover stable runtimes, partial/final runtime
    removal with an unchanged chunk hash, moved-and-edited modules, and the
    disabled CHUNK_ASSET fallback;
  • Copy matrix covers native watcher and Watchpack sibling creation, ancestor
    file/directory events, add/edit/remove-last/re-add, ignores,
    callback/template/permission bypass, and repeated compiler.run() without
    watcher provenance; it also covers 64 interleaved equal-priority forced
    destinations, mixed cache hit/miss ordering, POSIX asset keys on Windows,
    simple/nested [name]/[path] templates on both initial and unrelated
    rebuilds with an explicit no-backslash-key assertion, and successful-sibling
    caching across an errored glob recomputation. The
    watcher-backed assertion waits for its specific copied asset so queued
    startup events cannot create a false failure. A missing required glob
    reports exactly one diagnostic and recovers cleanly;
  • watcher unit/integration regressions cover an initially empty child, new
    grandchildren, recursive/non-recursive mode transitions, recursive-parent
    removal with a retained child, pruning overlapping descendants, and
    generation-aware pending-event delivery/drain/acknowledgement, callback
    teardown, uint32 generation wrap, and thousands of stale/retained sibling
    registrations with recursive children and near-prefix siblings;
  • native raw-event regressions cover create-only, simultaneous edit/create,
    ignored descendants, a 128-file lossless burst, remove, delayed
    create/delete, exact Watchpack-compatible aggregate parity, and rejection of
    stale/post-close callback generations;
  • dynamic/global-entry regressions cover option/filename changes, ordered
    import roots, unnamed global roots, named/global includes, original request
    attribution, unchanged duplicate-root reuse, and both shared and freshly
    captured filename/public-path functions across repeated rebuilds;
  • binding-entry regressions cover eager unnamed entries/named and global
    includes, ordinary named-entry laziness, and equal relative requests from
    distinct raw contexts for both addEntry and addInclude;
  • lazy/MultiCompiler and runtime regressions cover hook order, artifact
    dependency ordering, native and Watchpack coalesced file provenance without
    a duplicate dependent generation, canonicalization-safe native invalidation,
    the non-consuming paused-Watchpack signal and guaranteed in-flight release
    before close, initial JS/CSS chunks,
    worker chunks, full-hash filename templates (including extracted async CSS),
    compiler-prefix collision, and disposed factories across success, queued,
    and error paths; the preserved-factory case retains a closure created during
    apply and proves its post-idle disposed warning/no parent reattachment;
  • WebRunner constructs two isolated runners and proves console.warn
    overrides cannot cross-contaminate or mutate the process console;
  • the native lifecycle regression overlaps rewatch/pause/double-close, runs
    the 400-file/20-directory/five-watch/no-pause immediate-close stress shape,
    and proves that an immediately recreated watcher receives its full
    dependency set and first event. The test package explicitly declares its
    direct @rspack/binding dependency and lock entry instead of relying on
    incidental workspace resolution;
  • connection-wrapper GC regression discriminates the leak with forced GC;
    CSS/SRI/Rsdoctor cleanup is additionally covered by the compilation-map
    source regression and native plugin/HMR smoke;
  • focused validation is 54/54 base tests across ten files, 2/2 native
    watcher-generation tests, and 3/3 x5 repeated native/Watchpack
    coalescing/getInfo cases; the standalone correctness scenarios and focused
    Rust/core/watcher suites also pass. JS syntax and formatting are clean. The
    exact CI-produced WASI binding passes 10/10 focused Copy/coalescing/
    getInfo cases with Watchpack enabled; the native matrix passes 12/12 x3.
    The final CI-profile binding additionally passes 20/20 base
    Copy/lazy/watch cases across six files, 7/7 native-generation/raw/
    lifecycle cases across three files on each of Node 22/24 (54/54), and
    the six regenerated hot snapshots,
    including the 128-event raw burst, 1,000 overlapping lifecycle iterations,
    the 30-iteration
    no-pause/400-file close race, and immediate-recreate delivery. The
    earlier CI-profile worker/console matrix passes 349/349 on both
    Node versions with --maxConcurrency 4 --retry 0, and the final six preserved-
    factory runtime variants pass (18/18 selected assertions). With the
    repository-supported -t preserve-disposed-module-factories filter, the
    primary HMR cases take 121--207 ms in the final Node-22 run; the long-form
    --testNamePattern initializes unrelated hot cases and is not a valid
    focused timing. Focused watcher Clippy passes with warnings denied. The
    final commits pass git diff --check; the consolidated stack retains
    whitespace in one generated initial-chunks-hmr runtime snapshot.
    All CI passes on the exact ef90a0f head: Linux Node 20/24, WASM,
    macOS/Windows Node 22, all watcher platforms, Rust/Clippy, Dylint, bench,
    and binary size are green. Linux E2E passes 132/132 in 41.8s,
    resolving the preceding head's sole shared-stylesheet duplicate-link
    failure; Windows passes 8,931/8,931 enabled tests with 4 skipped,
    including all eleven CopyPluginCache tests and the simple/nested
    template-key regression. The maintained local CSS matrix and two
    regenerated CSS-recovery snapshots also pass. The native source is
    unchanged by the loader fix, so stripped size remains 69,831,832 B.
    The prior Rust-check CI failures (clippy::needless_borrow and
    clippy::from_iter_instead_of_collect in Copy) are fixed.

The small CodSpeed signals on the predecessor PRs were about 3% for
cold-stage microbenchmarks (create_chunk_assets and runtime_requirements;
the latter explicitly compared different runtime environments). The grouped
release A/B above does not reproduce them, and the HMR fast path returns
before doing work on a cold compilation.

The PR is intentionally kept as one handoff stack, with each topic small
enough to review or lift independently.

@matthewdavis-oai matthewdavis-oai marked this pull request as ready for review July 13, 2026 21:42
Copilot AI review requested due to automatic review settings July 13, 2026 21:42
Copilot AI reviewed Jul 13, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves incremental chunk-graph reuse during HMR by making BuildChunkGraphArtifact::can_skip_rebuilding_legacy compare dependency outgoings using the same per-block (root vs async block) semantics and ordering rules as CodeSplitter::prepare, avoiding false cache misses caused by async-block outgoings and dependency creation order.

Changes:

  • Update the legacy chunk-graph reuse predicate to (1) validate async block identity/options stability and (2) compare outgoings per root/async block with source_order-aligned ordering and filtering.
  • Expose CodeSplitter::prepared_blocks_map to pub(crate) so the reuse predicate can compare current vs cached async blocks.
  • Add a new watch regression case covering “unchanged leaf edit should reuse” vs “async chunk-name change must rebuild”.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
crates/rspack_core/src/artifacts/build_chunk_graph_artifact.rs Reworks the legacy reuse predicate to compare root + async block outgoings independently and validate async-block option stability.
crates/rspack_core/src/compilation/build_chunk_graph/code_splitter.rs Makes prepared_blocks_map accessible within the crate to support reuse checks.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/test.config.js Asserts (via stats/logging) that leaf-only edits reuse the chunk graph and async chunk-name changes rebuild.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/rspack.config.js Configures incremental buildChunkGraph + verbose logging for the regression fixture.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/0/index.js Initial entry module for the watch fixture.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/0/route.js Initial route module with sync imports, re-export, and a stable async import.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/0/leaf.js Initial leaf module content used to trigger a “no topology change” edit.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/0/terminal.js Stable dependency for leaf content.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/0/sync-first.js Sync dependency used to validate ordered sync import handling.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/0/sync-second.js Sync dependency used to validate ordered sync import handling.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/0/side-effect.js Side-effect module used to validate side-effect filtering behavior.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/0/lazy.js Async target module for the route’s dynamic import.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/1/leaf.js Step 1 update: leaf-only edit intended to reuse the chunk graph.
tests/rspack-test/watchCases/build-chunk-graph/async-block-outgoings/2/route.js Step 2 update: changes webpackChunkName to force a rebuild.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/rspack_core/src/artifacts/build_chunk_graph_artifact.rs Outdated
@codspeed-hq

codspeed-hq Bot commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 43 untouched benchmarks
⏩ 47 skipped benchmarks1

Comparing matthewdavis-oai:matthewdavis-oai/hmr-outgoing-cache (e92e185) with main (03080ba)

Open in CodSpeed

Footnotes

  1. 47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@matthewdavis-oai matthewdavis-oai force-pushed the matthewdavis-oai/hmr-outgoing-cache branch from 3b1bc3b to d0bb617 Compare July 14, 2026 18:46
@matthewdavis-oai matthewdavis-oai mentioned this pull request Jul 14, 2026
Closed
@matthewdavis-oai matthewdavis-oai mentioned this pull request Jul 14, 2026
Closed
matthewdavis-oai and others added 13 commits July 14, 2026 19:50
@matthewdavis-oai
perf(core): reuse chunk graph with stable async outgoings
d138a89
@matthewdavis-oai
test: format async outgoing watch fixture
91cc3c8
@matthewdavis-oai
perf(core): tighten chunk graph reuse and release connection wrappers
e8740f6
@matthewdavis-oai
perf(copy-plugin): reuse unchanged static copy patterns during watch
4710507
@matthewdavis-oai
fix(cache): release compilation-scoped plugin state on rebuild
8887a86
@matthewdavis-oai
perf(core): keep stable async and existing leaf chunk graphs
ce7ca20
@matthewdavis-oai
perf(hmr): skip unchanged chunks without dropping installed updates
f10f433
@matthewdavis-oai
fix(watch): preserve recursive contexts and copy invalidation
0aabc05
@matthewdavis-oai
fix(cache): keep compilation dependency deltas scoped
601ac45
@matthewdavis-oai
test(binding): guard module graph connection cleanup
182ab3c
@matthewdavis-oai @ScriptedAlchemy
fix(hmr): consolidate lazy compilation and runtime correctness
32d9804 
Port the final, CI-validated source and regression diff from web-infra-dev#14753 onto the consolidated HMR stack. This retains typed watch provenance, native event draining, initial-chunk/full-hash handling, and disposed-factory correctness.

Co-authored-by: ScriptedAlchemy <25274700+ScriptedAlchemy@users.noreply.github.com>
@matthewdavis-oai
fix(watch): keep dependent lazy generations and completed deltas correct
2ee3fd6 
Snapshot completed watch deltas before user callbacks can synchronously invalidate, and classify artifact-dependent MultiCompiler children as normal while preserving source-only lazy activation. Add focused artifact-chain, coalescing, and delta-delivery regressions.
@matthewdavis-oai
fix(watcher): keep native aggregate delivery lossless across generations
cd55890 
Allow the single in-flight aggregate to use a lossless TSFN queue, release teardown-only failed callbacks safely, and compare uint32 generations with wrap-aware serial arithmetic. Add pending-event restart and generation-wrap regressions.
@matthewdavis-oai matthewdavis-oai mentioned this pull request Jul 14, 2026
Open
@matthewdavis-oai matthewdavis-oai force-pushed the matthewdavis-oai/hmr-outgoing-cache branch from db15197 to 17d545c Compare July 14, 2026 21:23
@matthewdavis-oai matthewdavis-oai changed the title perf(core): reuse chunk graph with stable async outgoings perf(hmr): consolidate fast incremental updates and lazy-runtime correctness Jul 14, 2026
This was referenced Jul 14, 2026
Open
Open
Closed
@ScriptedAlchemy ScriptedAlchemy mentioned this pull request Jul 14, 2026
Open
@ScriptedAlchemy

ScriptedAlchemy commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

WASM Node 24 is deterministically failing both new native-watcher matrix rows (all 4 attempts timed out):

  • CopyPluginCache.test.js: watches the stable glob base and copies a newly populated sibling with native watcher
  • LazyCompilationArtifactChain.test.js: keeps dependent and coalesced file-backed generations normal with native watching

The corresponding Watchpack rows pass. rstest.config.mts already excludes NativeWatcher*.test.js under process.env.WASM; these mixed-mode files should omit only their nativeWatcher === true row when process.env.WASM, preserving their Watchpack coverage.

Failing job: https://github.com/web-infra-dev/rspack/actions/runs/29369339433/job/87210554619

@ScriptedAlchemy

ScriptedAlchemy commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

The binding-size failure is also real, not a runner flake: 17d545c is 69,864,600 bytes versus 69,782,680 on 03080ba, an 81,920-byte increase. The gate allows 51,200 bytes, so this is 30,720 bytes over the limit. The workflow's own report comment failed with a fork-token 403, which hid the delta from the PR timeline.

Failing job: https://github.com/web-infra-dev/rspack/actions/runs/29369339433/job/87212276014

This was referenced Jul 14, 2026
Open
Open
Open
@matthewdavis-oai matthewdavis-oai mentioned this pull request Jul 14, 2026
Open
This was referenced Jul 15, 2026
Open
Open
Open
@matthewdavis-oai
fix(hmr): keep extracted css reload single-owned
ef90a0f 
Avoid racing the extracted-CSS runtime with the loader-level reload while preserving the runtime:false fallback. Extend the incremental browser regression with repeated edits and exact request/link counts, and refresh the two affected hot snapshots.
@matthewdavis-oai
perf(copy-plugin): reuse unchanged patterns for lazy rebuilds
eb10e07 
Carry explicit lazy-watch provenance across the native boundary so an empty lazy activation can reuse static Copy results without making normal invalidations or repeated compiler.run calls stale. Preserve the existing Rust and three-argument binding APIs, add Perfetto attribution, and cover the 2,000-file and freshness regressions.
@matthewdavis-oai
fix(watcher): isolate stale native watch generations
e92e185 
Keep retained watcher handles, callbacks, and shims from pausing, draining, closing, purging, or injecting into a live native-watch generation. Acknowledge obsolete aggregates so delivery cannot stall and cover long-lived raw listeners, stale handles, and generation replacement.
@matthewdavis-oai matthewdavis-oai force-pushed the matthewdavis-oai/hmr-outgoing-cache branch from e6a1db8 to e92e185 Compare July 15, 2026 03:27
JSerFeng
JSerFeng reviewed Jul 15, 2026
View reviewed changes
Comment thread crates/rspack_core/src/dependencies_block.rs
Comment on lines +143 to +153
fn set_identifier_occurrence(
&mut self,
identifier: AsyncDependenciesBlockIdentifier,
occurrence: usize,
) {
let mut id = String::with_capacity(identifier.0.as_str().len() + "|occurrence=".len() + 20);
id.push_str(identifier.0.as_str());
write!(id, "|occurrence={occurrence}").expect("write to String should not fail");
self.id = id.into();
}

@JSerFeng JSerFeng Jul 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems that loc is indeed necessary for identifier here, as the loc may change at rebuild, and it should causing the AsyncDependenciesBlock change as well, because the error message and origin field depend on loc to display, but current implementation won't trigger change, as AsyncDependenciesBlockIdentifier during rebuild remains the same.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment